feat(search): add OpenAI search passthrough#175
Conversation
The Codex CLI's web-search tool POSTs a SearchRequest to `/alpha/search`
and feeds the returned `output` string back to the model. Serve it from
Floway's own configured web-search provider instead of proxying
chatgpt.com.
codex's `commands` is an object keyed by command kind
(`{search_query:[{q}], open:[{ref_id}], find:[{ref_id,pattern}], …}`) —
byte-for-byte the shape the Responses web_search shim already parses. So
extract the protocol-neutral execution engine (command parsing, provider
resolution, search/open/find execution, page cache, IR building, text
rendering, domain filtering) out of the Responses shim into a shared
tools/web-search/operations.ts, and build both surfaces on it: the shim
wraps each op into a web_search_call wire item; the Codex endpoint
concatenates the rendered text of every op into one output string.
Unimplemented command kinds (image_query, click, screenshot, finance,
weather, sports, time, response_length) surface as deterministic per-op
error text. Disabled/missing search config surfaces the same in-band
'provider not configured' text the shim uses. Request validated with a
zod schema mirroring codex-rs SearchRequest; usage accounted per API key.
Route tests through the codex namespace: auth, zod schema rejection (non-object commands, bad search_context_size), search/open/find command execution and rendered output, per-key usage accounting, unsupported command text, domain-filter blocking, empty commands, and disabled / missing-credential provider states surfaced as in-band output.
Codex derives its remote web-search URL from different base URLs across auth and provider modes. Mount the same authenticated, schema-validated handler at /alpha/search and /v1/alpha/search in addition to the generated config's /azure-api.codex/alpha/search path.
Parameterize routing, missing/invalid auth, and schema rejection across the namespaced, root, and /v1 alpha-search paths so aliases cannot drift from the primary handler.
Route /alpha/search to the gateway in Vite development, Cloudflare Static Assets, and the Docker nginx topology. /v1/alpha/search is already covered by each topology's existing /v1 prefix rule.
|
Claude Code agent update: root compatibility aliases are being added. Codex joins |
Codex recognizes a custom provider as eligible for its remote web.run tool when the provider config carries a non-empty x-openai-actor-authorization header. Treat that dashboard-generated marker as gateway-owned identity and strip it before provider dispatch so it reaches Floway's routing decision but never leaks to an LLM upstream.
Codex gates the remote web-search extension on either an OpenAI provider identity or a non-empty x-openai-actor-authorization provider header. Keep the provider key/name as floway/Floway so the wider is_openai behavior stays disabled, and add a non-secret marker header that enables only the actor-auth compatibility gates. The gateway scrubs the marker before upstream dispatch. Cover the generated TOML so the provider identity, namespaced base URL, and compatibility marker cannot drift.
The actor-authorization marker that enables remote web.run also makes Codex consider its remote image-generation extension. Floway does not serve the provider-relative images/generations path, so explicitly disable that feature while retaining the marker for web search. Cover and document the generated feature block.
Keep the inbound-header scrub list alphabetic, describe alpha-search output in the parser's canonical operation order rather than JSON key order, and make the shared truncated-page hint neutral across Responses and Codex callers.
|
Claude Code agent correction: the generated |
Keep the generated Codex provider config faithful to Floway: do not inject x-openai-actor-authorization, do not special-case that vendor header in the gateway scrubber, and do not disable image generation. Remove the associated gating claims and retain only the actual alpha-search route behavior in the documentation.
…aa14b68090a # Conflicts: # packages/gateway/src/data-plane/chat/responses/interceptors/server-tools/web-search.ts # packages/gateway/src/data-plane/codex/routes.ts
Add callAlphaSearch to the provider contract. Codex dispatches to the ChatGPT subscription endpoint with its OAuth account, custom dispatches to /v1/alpha/search with static auth, and other provider kinds reject the unsupported call.
Persist a default-off OpenAI search passthrough selection with upstream and model. Add the Settings toggle and scoped Codex/Custom upstream and chat-model pickers while preserving the existing general search provider as the default.
Resolve the configured upstream and model without fallback. Raw Codex alpha-search requests relay the selected provider response, while the existing Responses web-search shim adapts commands and structured results through the same provider call.
Thread the default-off passthrough shape through existing search fixtures and finish the merged web-search filter and Azure unsupported-provider paths.
Verify custom and Codex provider wires, raw alpha-route relay, Responses shim adaptation, persisted configuration, and the Codex/Custom-only settings picker.
Type the settings component against its minimal upstream picker shape, handle the nullable model store at the page boundary, and apply workspace lint rules to the alpha dispatcher path.
Keep search configuration export and import fixtures aligned with the OpenAI search passthrough settings.
Publish the required OpenAI search passthrough settings under transfer format v10 so incompatible payload shapes never share a version number.
Keep Codex request identity and optional metadata coherent, document the ChatGPT search path, and route Custom alpha search through the existing endpoint override policy.
Make search configuration writes typed, remove lossy post-validation normalization, enforce the stored boolean domain, and keep the dashboard selector invariants explicit.
Use a callable alpha dispatcher, remove test-only compatibility exports, preserve cancellation through local search execution, and align comments and references with both local and passthrough modes.
Keep the README focused on public Floway endpoints and omit the internal Codex compatibility namespace.
Rebuild passthrough responses with mutable headers before returning through Hono so Custom-to-Floway alpha-search chains preserve the upstream stream, status, and headers on Node.
State that the selected upstream must support alpha search and that Anthropic Messages remains on the general search provider.
|
Codex agent update: the ChatGPT Codex standalone-search wire probe is complete. This comment embeds the full publishable experiment archive; it is self-contained and does not depend on a gist or external host. Evidence: 349 Important findings for this PR:
Redaction removed all access/refresh/ID tokens, account and user identifiers, emails, personal paths, proxy configuration, IPs, opaque encrypted payloads, account-plan headers, and full fetched page bodies. Successful responses retain types, key sets, counts, lengths, and diagnostic classifications. The supplied credential's refresh token was never read or invoked by the online runners. |
Fetch exposes a decoded body while retaining upstream content-encoding and content-length. Rebuilding the alpha response with those stale headers makes a downstream Floway or Codex client decode plain JSON again and fail with a terminated stream.\n\nStrip hop-by-hop, stale representation-framing, and upstream cookie headers while retaining content type and vendor metadata. Cover the shared fetched-response relay behavior directly.
Live alpha-search results are opaque UI metadata and omit the hosted web_search snippet field. The existing compatibility filter therefore dropped every real result before silently falling back to the rendered output.\n\nKeep the upstream output as the lossless model-facing payload and emit one clearly synthetic Responses result when that surface needs result-shaped data. Raw alpha routes continue relaying the upstream response untouched.
|
Codex agent update: the isolated Codex 2×3 end-to-end matrix is complete on the final PR head.
Each cell used its own Routing evidence:
The first cascade attempt exposed a real response-relay bug: Node Fetch had decoded upstream gzip bytes but Floway relayed the stale The live alpha study also showed that official structured results omit hosted-search Final verification:
|
Move the alpha route, dispatcher, Responses adapter, response relay, and their tests into one web-search/alpha-search slice. The Codex route module now only mounts the shared handler, and the fetched-response helper no longer implies gateway-wide reuse.
Give the alpha-search data-plane surface ownership of its paths, validation, and handler while keeping provider execution, upstream dispatch, response adaptation, and relay behavior under tools/web-search/alpha-search. Codex routes now contain only the compatibility namespace they actually mount.
Let the general alpha-search data plane own only the root and /v1 aliases. Codex alone declares its special provider-relative path and reuses a path-agnostic route registrar. Hosted search remains coupled only to the tools/web-search implementation layer.
Summary
Add a default-off OpenAI search passthrough that lets Responses hosted web search and Codex standalone
web.runuse a selected Codex or Custom upstream with/alpha/searchsupport. The existing Tavily, Microsoft Grounding, or Jina configuration remains the default search backend, Anthropic Messages is unaffected, the selected upstream must be visible to the calling API key, and passthrough never falls back to another search provider.Persist the toggle, upstream, and model in search settings, expose the dependent upstream/model controls in the dashboard, and carry the required shape through strict import/export format v10. The provider contract gains
callAlphaSearch: Codex forwards to the ChatGPT subscription endpoint, Custom uses its configurable OpenAI-compatible path, and unsupported provider kinds reject the operation.Serve the general data-plane aliases at
/alpha/searchand/v1/alpha/search; the special/azure-api.codex/alpha/searchpath remains owned and mounted only by the Codex namespace. Shared alpha execution, upstream selection, Responses adaptation, and relay behavior live undertools/web-search/alpha-search, so the hosted web-search shim depends only on the reusable web-search implementation rather than on HTTP routes.Local mode parses and executes supported alpha commands through the general search backend. Passthrough mode relays the upstream status, decoded body stream, content type, and representation-safe metadata while removing stale content coding/length, hop-by-hop headers, and upstream cookies. Raw alpha JSON remains intact. Responses preserves the alpha endpoint's model-facing
output; because alpharesultsare opaque UI metadata and do not use the hosted-searchsnippetshape, the adapter emits an explicitly synthetic result only when the Responses surface needs result-shaped data.Account affinity for a future pool of Codex accounts remains a TODO. This change intentionally adds neither fallback nor a loop-prevention protocol.
Evidence
The live-wire study covered 349 ChatGPT Codex
/alpha/searchrequests and six Responses hosted-search comparisons. The redacted specification, probe runners, and per-case structural reports are attached in this PR comment.Test plan
pnpm run test— 358 files / 4,359 testspnpm run lintpnpm run typecheck— 19 workspace projectsHOME/CODEX_HOMEpairs across hosted/client tools × general/official/cascade routes — all six cells completed one web search and returned the expected IANA URL